Recognition Networks for Approximate Inference in BN20 Networks

A recognition network is a multilayer per­ ception (MLP) trained to predict posterior marginals given observed evidence in a par­ ticular Bayesian network. The input to the MLP is a vector of the states of the eviden­ tial nodes. The activity of an output unit is interpreted as a prediction of the posterior marginal of the corresponding variable. The MLP is trained using samples generated from the corresponding Bayesian network. We evaluate a recognition network that was trained to do inference in a large Bayesian network, similar in structure and complex­ ity to the Quick Medical Reference, Decision Theoretic (QMR-DT) network. Our network is a binary, two-layer, noisy-OR (BN20) net­ work containing over 4000 potentially observ­ able nodes and over 600 unobservable, hidden nodes. In real medical diagnosis, most ob­ servables are unavailable, and there is a com­ plex and unknown process that selects which ones are provided. We incorporate a very ba­ sic type of selection bias in our network: a known preference that available observables are positive rather than negative. Even this simple bias has a significant effect on the pos­ terior. We compare the performance of our recogni­ tion network to state-of-the-art approximate inference algorithms on a large set of test cases. In order to evaluate the effect of our simplistic model of the selection bias, we eval­ uate algorithms using a variety of incorrectly modelled selection biases. Recognition net­ works perform well using both correct and incorrect selection biases. • also affiliated with Department of Brain and Cogni­ tive Sciences at MIT